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Abstract — We consider the problem of efficient on-line anomaly 
detection in computer network traffic. The problem is ap- 
proached statistically, as that of sequential (quickest) changepoint 
detection. A multi-cyclic setting of quickest change detection 
is a natural fit for this problem. We propose a novel score- 
based multi-cyclic detection algorithm. The algorithm is based on 
the so-called Shiryaev-Roberts procedure. This procedure is as 
easy to employ in practice and as computationally inexpensive 
as the popular Cumulative Sum chart and the Exponentially 
Weighted Moving Average scheme. The likelihood ratio based 
Shiryaev-Roberts procedure has appealing optimality properties, 
particularly it is exactly optimal in a multi-cyclic setting geared 
to detect a change occurring at a far time horizon. It is therefore 
expected that an intrusion detection algorithm based on the 
Shiryaev-Roberts procedure will perform better than other de- 
tection schemes. This is confirmed experimentally for real traces. 
We also discuss the possibility of complementing our anomaly 
detection algorithm with a spectral-signature intrusion detection 
system with false alarm filtering and true attack confirmation 
capability, so as to obtain a synergistic system. 



I. Introduction 

The Internet has never been a safe place and designing auto- 
mated and efficient techniques for rapid detection of computer 
network anomalies (e.g., due to intrusions) never ceased to be 
a topical problem in cybersecurity [ 1 1. Many existing anomaly- 
based Intrusion Detection Systems (IDS-s) operate by applying 
the machinery of statistics to comb through the passing traffic 
looking for a deviation from the traffic's normal profile 12- 
151 . By way of example, the Sequential Probability Ratio 
Test (SPRT) Q, the Cumulative Sum (CUSUM) chart (8), 
and the Exponentially Weighted Moving Average (EWMA) 
inspection scheme [9] are the de facto "workhorse" of the 
community. The CUSUM and EWMA methods come from the 
area of sequential changepoint detection, a branch of statistics 
concerned with the design and analysis of a fastest way to 
detect a change (i.e., an anomaly) in the state of a phenomenon 
(time process) of interest iflOl . ifTTI . 

Yet another changepoint detector popular in statistics is 
the Shiryaev-Roberts (SR) procedure [ 12|-| 14|. Though prac- 
tically unknown in the cybersecurity community, the SR 
procedure is as computationally simple as the CUSUM chart 
or the EWMA scheme. However, unlike the latter two, the SR 
procedure is also the best one can do (i.e., exactly optimal) 



in a certain multi-cyclic setting 0311 , a natural fit in the 
computer network anomaly detection context. The aim of 
this work is to offer a novel multi-cyclic anomaly detector 
using the SR procedure as the prototype. Due to the exact 
multi-cyclic optimality of the SR procedure, the proposed 
algorithm is expected to outperform other detection schemes, 
in particular the multi-cyclic CUSUM procedure. We confirm 
this experimentally using real data. 

The remainder of the paper is organized as follows. Sec- 
tion [ll] provides an introduction to the subject of changepoint 
detection. In Section [TTTJ we present our anomaly detection 
algorithm. In Section |IV| we illustrate our algorithm at work. 
In Section|V] we comment on how to improve the performance 
of the algorithm. Lastly, Section [VT1 draws the conclusions. 

II. Quickest Changepoint Detection 

Quickest changepoint detection is a study of techniques to 
detect a change ("disorder") in the state of a time process, 
usually from "normal" to "abnormal"; inference about the 
process' current state is made from a series of quantitative 
random observations (e.g., measurements corrupted by noise). 
The sequential setting assumes the series is amassed one at a 
time, and so long as the recorded data behavior suggests the 
process is in its "normal" state it is let to continue. However, if 
the observations hint that the process' state may have switched 
to "abnormal", one's aim is to detect the true change as quickly 
as possible for a given risk associated with false alarms, so that 
an appropriate response can be provided in a timely manner. 
The time instance at which the state of the process changes 
is referred to as the changepoint, and the challenge is that 
it is not known in advance. This is known as the sequential 
(quickest) changepoint detection problem. For lack of space, 
from now on we will focus only on the basic iid version of 
this problem; a general non-iid case is surveyed, e.g., in fl6l . 

Suppose one is able to sequentially collect a series 
of independent random observations, {X n } n ^i, such that 
Xi, . . . , X v are each distributed according to a known prob- 
ability density function (pdf) /, while X v +i, X u+ 2, ■ ■ ■ each 
adhere to a pdf g ^ /, also known. The time index v (i.e., 
the changepoint) is assumed unknown non-random number; 
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for cases that regard v as a random variable, see, e.g., [12|, 
|f]~3l . One's aim is to detect that the observations' common 
distribution has changed. The challenge is to do so with as few 
observations as possible following the changepoint, subject to 
a tolerable limit on the risk of making a false detection. 

Statistically, the problem is to sequentially differentiate 
between the hypotheses "Hk : v = k, ^ fc < oo (i.e., that the 
data {X n }„^i change their statistical profile at time instance 
v = k, ^ k < oo) and Hoo : v = oo (i.e., that no change 
ever occurs). To test Hk against H.^ one first constructs the 
corresponding likelihood ratio, which for the iid scenario has 
the form 

- n( X \ 

A fc: „= J] A„ where A,- = ^4-, 

j=k+l ^ j ' 

and it is understood that Afc : „ = 1 for k ^ n. 

Next, as each new observation becomes available to test 
the hypotheses, the sequence {Afc :n }i^fc^„ is turned into 
a detection statistic. To this end, one can either use the 
maximum likelihood principle or the (generalized) Bayesian 
approach. In the former case the corresponding detection 
statistic is 

V n = max Ak-.n, n > 1, (1) 

i.e., the famous CUSUM statistic. The Bayesian statistic 
depends on the changepoint's prior distribution. As in our case 
the changepoint, v, is assumed unknown, the corresponding 
quasi-Bayesian (or generalized Bayesian) detection statistic 
can be defined as 

n 

R n = ^ A fc:n , n > 1. 

k=l 

One can view {R n } n ^i as being the average of the sequence 
{Afc :n }i^fc^„ with respect to an (improper) uniform prior 
distribution imposed on v\ see, e.g., fl2l . |[T3l . [16|-[18|. 

Once the detection statistic is chosen, it is supplied to 
an appropriate sequential detection procedure. A detection 
procedure is a stopping time, T, which is a function of the 
observed data, {l„}„^i. The meaning of T is that after 
observing X\ , . . . , Xt it is declared that the change is in 
effect. That may or may not be the case. If it is not, then 
T ^ v, and it is said that a false alarm has been sounded. 

Henceforth, let Pi,(-) and Poo(') denote the probability 
measures, respectively, when the change occurs at time instant 
^ v < oo, and when no change ever occurs. Likewise, let 
Ej,[-] and E^-] be the corresponding expectations. 

Lorden |[T9l suggested to measure the risk of raising a 
false alarm via the Average Run Length (ARL) to false alarm 
ARL(T) = EoofT] and showed that the CUSUM procedure 
has certain minimax properties in the class of detection pro- 
cedures 

A( 7 ) = {T: ARL(T)^ 7 } 

for which the ARL to false alarm is no "worse" than the 
desired a priori chosen level 7 > 1. See also Moustakides [20 1 
who proved that CUSUM is in fact strictly minimax with 
respect to Lorden's criterion for every 7 > 1. 



A practically appealing way to measure the detection speed 
is Pollak's ETI "worst-case" (Supremum) Average Delay to 
Detection (ADD), conditional on a false alarm not having been 
previously sounded, i.e., 

SADD(T) = max E k \T - k\T > k}. 

0<fc<oo 

The minimax quickest changepoint detection problem is to 
find T op t G A (7) such that 

SADD(T OD t) = inf SADD(T) for all 7 > 1. 

TSA( 7 ) 

To date, this problem remains open, and only asymptotic (as 
7 — > 00) solutions have been obtained [21], 11221 . 

The CUSUM chart [8| has been popular in many areas 
of engineering and computer science, including cybersecurity. 
It iteratively maximizes the log-likelihood ratio (LLR) with 
respect to the changepoint v, and stops once the maximum 
exceeds a certain threshold. More specifically, the CUSUM 
procedure is based on the statistic W n = max{0, log V n }, 
where V n is defined in which is computed recursively 

W n = max{0, W n _i + £„}, n > 1, W Q = 0. 

Here L n — logA„ is the LLR. The corresponding stopping 
rule is 

C h = mm{n ^ 1 : W n ^ h}, 

where h > is a detection threshold preset so as to achieve 
the desired level of false alarms 7 > 1, and thus guarantee that 
Ch s A (7). This can be achieved by setting h = h 7 ^ log 7, 
since ARL(Ch) ^ e h for any h > fl9l . For large values of 
7 more "careful" selection of h is possible lPT7l . 

Consider now a context in which it is of utmost importance 
to detect the change as quickly as possible, even at the expense 
of raising many false alarms (using a repeated application 
of the same stopping rule) before the change occurs. Put 
otherwise, in exchange for the assurance that the change will 
be detected with maximal speed, we agree to go through 
a "storm" of false alarms along the way (the false alarms 
are ensued from repeatedly applying the same detection rule, 
starting from scratch after each false alarm). This scenario is 
shown in Figure Q] 

Formally, let T\ , T2, . . . be sequential independent repeti- 
tions of the stopping time T, and let TJ = Ti + T 2 H h 2), 

j ^ 1, be the time of the j-th alarm. Define l v = min-jj ^ 
1 : 7j > v}. In other words, 7i„ is the time of detection of 
a true change that occurs at v after l v — 1 false alarms have 
been raised. Write 

STADD(T) = lim E V [T, U - v] 

for the limiting value of the average delay to detection referred 
to as the Stationary Average Delay to Detection (STADD). The 
multi-cyclic changepoint detection problem is to find T op t £ 
A(7) such that 

STADD (T opt ) = t inf ^ STADD (T) for every 7 > 1. 

This formulation is instrumental in detecting a change that 
takes place in a distant future (i.e., v is large), and is preceded 
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(a) An example of the behavior of a process of interest with a change in mean at time v. 
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(b) Typical behavior of the detection statistic in the multi-cyclic mode. 
Fig. 1. Multi-cyclic changepoint detection in a stationary regime. 



by a stationary flow of false detections. Such scenarios are 
a commonplace in the area of computer network anomaly 
detection. 

As has been shown by Pollak and Tartakovsky 031 . the so- 
called Shiryaev-Roberts (SR) procedure lfl3ll . lfl4l is exactly 
optimal for every 7 > 1 with respect to the stationary average 
detection delay STADD(T). Thus, in the multi-cyclic setting 
the SR procedure is a better alternative to the popular CUSUM 
and EWMA schemes. 

The SR rule stops at time instance 

Sa = min{n ^ 1 : R n ^ A}, 
where the SR statistic is given by the recursion 

R n = (1 + R n -i) A„, n > 1, R = 0. 

Here A > is a detection threshold set a priori so as to 
ensure Sa G A (7) for a desired 7 > 1. It can be easily 
shown (23] that ARL(Sa) ^ A for all A > 0, so choosing 
the detection threshold as A 7 = 7 will guarantee Sa £ A (7). 
A very accurate asymptotic approximation ARL(iSyi) ~ A/v, 
A — ► 00 is also possible, where < v < 1 is a constant which 
is a subject of renewal theory. See, e.g., (23). 

III. Transition to Cybersecurity 

The above somewhat abstract introduction to sequential 
changepoint detection is straightforward to put in the context 
of anomaly detection in computer network traffic. As network 
anomalies typically take place at unknown points in time 



and entail changes in the traffic's statistical properties, it is 
intuitively appealing to formulate the problem of computer 
network anomaly detection as that of a quickest changepoint 
detection: to detect changes in the statistical profile of network 
traffic as rapidly as possible, while maintaining a tolerable 
level of the risk of making a false detection. 

It is common that in practice neither pre- nor post-anomaly 
distributions are known. As a result, traffic's pre- and post- 
anomaly profile is poorly understood, and one can no longer 
rely on the likelihood ratios. Hence, an alternative approach 
is required. Let us first consider a typical behavior of the 
CUSUM and SR statistics. As long as the observed sequence 
{X n } n ^i is in the normal mode, the detection statistics 
{Bnjn^i an d {Rn}n^i behave as if they were "afraid" of 
approaching the detection thresholds h and A respectively 
(although it is still possible that the thresholds will be crossed, 
in which case a false alarm will be raised). However, as soon 
as X„ + i - the first data point affected by an anomaly - is 
recorded, the behavior of W n and R n changes completely, so 
that they now eagerly try to hit the thresholds. Formally, this 
means that E^^,,] < and E^yCn] > 0, v < n. That is, the 
detection statistic has a negative drift under the normal regime, 
and a positive drift in an anomaly situation. A typical behavior 
of the detection statistic in pre- and post-change regimes is 
shown in Figure [2] 

Consider now the following score-based modification of the 
SR procedure 

R n = (1 + R n -i)e s ", n > 1, = 
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(b) Two possible terminal decisions: either a false alarm (dashed), or a correct 
but delayed detection (solid). 

Fig. 2. Typical run of the detection statistic. 



with the corresponding stopping time being 

Sa — min{n ^ 1 : R n ^ ^4}, 

where A > is an a priori chosen detection threshold. 
Similarly for CUSUM, 

W n =mzx{0,W n -i + S n }, n^l, W n = 
with the corresponding stopping time being 

C h = min{ra ^l:W n >h}, h > 0. 

Here S n (Xi, . . . , X n ) are the selected score functions. 
Clearly, so long as 

E oo [S n (X 1 ,...,X n )]<0 and E v [S n (X u . . . , X n )] > 0, 

for all v ^ 0, the SR and CUSUM detection procedures 
designed using such score functions in place of the likelihood 
ratio will work, though they will not be optimal anymore. 
Their behavior will be similar to that shown in Figure [2] Score 
functions S n can be chosen in a number of ways and each 
particular choice depends crucially on the expected type of 
change. In the applications of interest, the detection problem 
can be usually reduced to detecting changes in mean values 
along with variances (mean and variance shifts). 
Let 

Moo = [X n ] , ale = Varoo [X n ] 



and 



H = Eo[X n ], a 2 =Var [X„ 



denote the pre- and post-anomaly mean values and variances, 
respectively. Write Y n = (X n — /i o)/o'oo for the centered and 
scaled observation at time n. In the real-world applications 



the pre-change parameters /i^ and a 2 ^ are estimated from 
the training data and periodically re-estimated due to the non- 
stationarity of network traffic at large time-scales. We suggest 
the score S n of the linear-quadratic form 



S n (Yn) — C\Y n + C2Y n — C3 



(2) 



where C\, Gi and C3 are positive design numbers assuming 
for concreteness that the change leads to an increase in both 
mean and variance. In the case where the variance either does 
not change or changes relatively insignificantly compared to 
the change in mean, the coefficient C2 may be set to zero. 
In the opposite case where the mean changes only slightly 
compared to the variance, we take C\ = 0. The first case 
appears to be typical for many cybersecurity applications, 
for example for ICMP and UDP Denial-of-Service (DoS) 
attacks (see [4], [5 | where the linear score-based CUSUM has 
been proposed). However, in certain cases, such as the one 
considered below in Section IIVI both the mean and variance 
change quite significantly. 

Note that the score given by (f2]i with 

C^Sq 2 , C 2 = (l-q 2 )/2, C 3 =6 2 q 2 /2-logq, (3) 

where q = CToo/ct, S = (fi — ^, 00 )/o'oo< is optimal if pre- and 
post-change distributions are Gaussian with known putative 
values fi and a 2 . This is true because in the latter case S n is 
the log-likelihood ratio. If one believes in the Gaussian model 
(which sometimes is the case), then selecting q = qo and 
S = So with some design values qo and Sq provides reasonable 
operating characteristics for q < qo and S > 5q and optimal 
characteristics for q — qo and S = 5q, However, it is important 
to emphasize that the proposed score-based SR procedure does 
not assume that the observations have Gaussian pre- and post- 
change distributions. 

Further improvement may be achieved by using either mix- 
tures or adaptive versions with generalized likelihood ratio- 
type statistics lfT9l <23l. 

Based on the previous reasoning (see Section HH we expect 
the multi-cyclic score-based SR procedure to perform better 
than the analogous CUSUM procedure. 

IV. A Case Study 

We now present the results of testing the proposed de- 
tection algorithms on a real Distributed DoS (DDoS) at- 
tack, namely, SYN flood attack. The aim of this attack 
is to congest the victim's link with a series of SYN re- 
quests so as to have the victim's machine exhaust all of 
its resources and stop responding to legitimate traffic. This 
kind of an attack clearly creates a volume-type anomaly 
in the victim's traffic flow. The data is courtesy of the 
Los Angeles Network Data Exchange and Repository (LAN- 
DER) project (see http://www.isi.edu/ant/lander). Specifically, 
the trace is flow data captured by Merit Network Inc. 
(see http://www.merit.edu). The attack is on a University of 
Michigan IRC server. It starts at roughly 550 seconds into 
the trace and has a duration of 10 minutes. The attacked IP 
is anonymized to 141.213.238.0. Figure [3] shows the number 
of attempted connections or the connections birth rate as a 
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function of time. While the attack can be seen to the naked 
eye, it is not completely clear when it starts. In fact, there 
is a spike in the data (fluctuation) before the attack. Also, 
controlling the false alarm rate with an automatic detection 
system is a challenge. 




Fig. 3. SYN flood attack: number of attempted connections. 

We used the number of connections during 20 msec batches 
as the observations X n . We estimated the connections birth 
rate average and variance for legitimate traffic and for attack 
traffic; in both cases, to estimate the average we used the usual 
sample mean, and to estimate the variance we used the usual 
sample variance. For legitimate traffic, the average is about 
Moo = 1669.09 connections per 20 msec, and the standard 
deviation is in the neighborhood of = 113.884 connections 
per 20 msec. For attack traffic, the numbers are fi = 1887.56 
and a — 218.107, respectively. We can now see the effect of 
the attack: it leads to a considerable increase in the mean and 
standard deviation of the connections birth rate. 

We now perform a basic statistical analysis of the con- 
nections birth rate distribution. Figure [4] shows the empirical 
densities of the connections birth rate for legitimate and attack 
traffic. It so happens that for given data, legitimate traffic 
appears to resemble the Gaussian process. However, for attack 
traffic, the distribution is not as close to Gaussian. We have 
implemented the score-based multi-cyclic SR and CUSUM 
procedures with the linear-quadratic score |0. When choosing 
the design parameters we assume the Gaussian model for 
pre-attack traffic, which agrees with the conclusions drawn 
above following the basic statistical analysis of the data. 
Thus, parameters C±,C2, and C3 are chosen according to 
formulas (f3j) with qo = q » 0.52 and to allow for detection 
of fainter attacks 6q ~ 1.5 (versus the estimated attack value 
5 w 1.9). We set the detection thresholds A « 1.9 x 10 3 
and h « 6.68 so as to ensure the same level of ARL at 
approximately 500 samples (i.e., 10 sec) for both procedures. 
The thresholds are estimated using Monte Carlo simulations 
assuming the empirical pre-change distribution learned from 
the data. Specifically, we took 10 5 samples from the empirical 
pre-change distribution and simulated the behavior of the 
respective detection statistics and procedures while adjusting 
the thresholds until observing the desired ARL. 
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(a) Legitimate (pre-attack) traffic. 
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(b) Attack traffic. 

Fig. 4. SYN flood attack: connections birth rate pdf with a Gaussian fit. 
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Fig. 5. SYN flood attack: long run of the Shiryaev-Roberts procedure; 
logarithm of the SR statistic vs time. 



The detection process is illustrated in Figure [5] and Figure [6] 
Figure [5] shows a relatively long run (taking into account the 
sampling rate 20 msec) of the SR statistic with several false 
alarms and then the true detection of the attack with a very 
small detection delay (at the expense of raising many false 
alarms along the way). Recall that the whole idea of this 
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(a) By the SR procedure 
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(b) By the CUSUM procedure 
Fig. 6. Detection of the SYN flood attack by the SR and CUSUM procedures. 

paper is to set the detection thresholds low enough in order 
to detect attacks very quickly with minimal delays, which 
unavoidably leads to multiple false alarms prior to the attack 
starts. These false alarms should be filtered by a specially 
designed algorithm, as has been suggested in lfl5l and will be 
further discussed in Section [V] 

Figure |6(a)| shows the behavior of the logarithm of the 
SR statistic shortly prior to the attack and right after the 
attack starts till its detection, which happens when the statistic 
crosses the threshold. Figure |6(b)| shows the same for the 
CUSUM statistic. We see that both procedures successfully 
detect the attack with very small delays, though at the expense 
of raising false alarms along the way, as shown in Figure [5] 
and discussed above. For both procedures we observed ap- 
proximately 7 false alarms per 1000 samples. The detection 
delay for the repeated SR procedure is roughly 0.14 seconds 
(or 7 samples), and for the CUSUM procedure the delay is 
about 0.21 seconds (or 10 samples). Thus, the SR procedure 
is better, as expected. 

V. Further Discussion 

Since in real life legitimate traffic dominates, the idea of 
comparing various anomaly-based IDS-s using the multi-cyclic 



approach and the stationary average detection delay is a natural 
fit for cybersecurity applications. However, it is worthwhile 
to remark on a possible way to enhance the potential of 
changepoint detection techniques as applied to cybersecurity. 
Any changepoint detection method is subject to the following 
drawback: instantaneous detection is not an option, unless the 
false alarm risk is high. Hence, though changepoint detection 
schemes are computationally inexpensive, in practice, employ- 
ing one such scheme alone may not be a good idea, since it 
will be overflowed with false alarms. The simplest solution is 
to increase detection thresholds dramatically, but this will lead 
to an increase of the detection delay. 

Here comes an interesting opportunity: What if one could 
combine changepoint detection techniques with others that 
offer very low false alarm rate, but are too heavy to use at 
line speeds? Do such synergistic anomaly detection systems 
exist, and how can they be integrated? 

As an answer, consider complementing a changepoint 
detection-based anomaly detector with a flow-based signature 
IDS that examines the traffic's spectral profile. For an example 
of such signature-flow-based method, see, e.g., |24| |27|. The 
principal idea is to employ the Fourier transform to obtain the 
corresponding spectral characteristics of the passing traffic. 
This idea can be used in conjunction with the changepoint 
detection-based anomaly detector for both rejection of false 
alarms and confirmation of true detections. Higher computa- 
tional complexity of the spectral-signature based detector is 
compensated by the preliminary changepoint anomaly based 
algorithm; the latter triggers the former only when there 
is a suspicion of an anomaly may be taking place in the 
network link of interest. For practical purposes the mean time 
between false alarms of the changepoint based anomaly IDS 
can be taken as small as a few seconds, as it was in the 
experiments presented in the previous section. We believe that 
such an alliance of the changepoint anomaly- and the spectral- 
signature-based detectors can significantly improve the whole 
system's overall performance reducing the false alarm rate to 
the minimum and at the same time guaranteeing very small 
detection delays. 

VI. Conclusion 

We addressed the problem of rapid anomaly detection in 
computer network traffic. Approaching the problem statisti- 
cally, namely, as that of sequential changepoint detection, we 
proposed a new anomaly detection method. The method is 
based on the multi-cyclic (repeated) Shiryaev-Roberts detec- 
tion procedure where the likelihood ratio is replaced with 
the linear-quadratic score. This is done because in real-world 
network security applications both pre-attack and post-attack 
distributions are different from hypothesized distributions such 
as Gaussian or Poisson. Like many changepoint detection 
schemes, our method is also of practically no computational 
complexity and easy to implement. However, what distin- 
guishes the SR procedure is its exact multi-cyclic optimality 
in a simple change detection problem where densities are 
known, a property that such techniques as the SPRT, the 
CUSUM chart, or the EWMA scheme lack. Hence, one may 
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conjecture that the score-based SR detection algorithm is 
a better cyber "watchdog". To support this conjecture, we 
conducted a case study using a real SYN flood attack. The 
score-based multi-cyclic SR algorithm outperformed the multi- 
cyclic CUSUM procedure. Lastly, as a possible improvement 
of any changepoint detection-based anomaly detector, we 
proposed to complement the latter with a signature-based 
spectral IDS. This approach will allow to filter false alarms 
reducing the false alarm rate to a minimum and simultaneously 
guaranteeing prompt detection of real attacks. 
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